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Executive Summary 


This report analyzes the effects of implementing a personalization for academic and social emotional 
learning (PASL) program in selected Broward County high schools. PASL was implemented with a 
continuous improvement model promoting deliberate adaptations to class- and school-level 
conditions. PASL practices include goal setting, use of data, educator teams, and points of contact 
between teachers and students. The implementation of PASL started at three high schools in 2014- 
2015 (the “innovation year”) and expanded to an additional five high schools in the 2015-2016 “scale- 
out year.” Because of the nature of the adaptive continuous improvement, there were many 
differences in implementation and treatment practices across schools. As a result of these 


differences, we analyze program effects both collectively and on a school-by-school basis. 


Overall, the effects of PASL were quite varied across schools and outcomes. Effects of PASL on test 
scores were especially variable. Only Pleasant Valley exhibited test score gains across all subjects, 
and several of the scale-out schools experienced drops in test scores when PASL was employed. 
Unfortunately, given the variation in treatment and fidelity of implementation across schools and 
over time, it is hard to determine why the estimated impacts of PASL were so heterogeneous. The 
generally positive impacts on non-test-score outcomes at Spring Gardens could be due to the way in 
which PASL was implemented there or to some other unique conditions at the school during the 
period of implementation. A full understanding of PASL’s impacts on students would require an 
experiment in which schools cannot self-select into the program and the intervention is implemented 


consistently across all schools. 


The most consistent evidence of an impact of PASL is a reduction of major disciplinary incidents in 
the innovation schools, although there was still a good deal of variation. Among the group of 
innovation schools, Spring Gardens experienced improvements in student behavior in both years as 
a result of implementing PASL. The gains at Spring Gardens were quite substantial, at about -0.17 
incidents per student for ninth graders. Given the pre-intervention average incidents per student in 
innovation schools equaled 0.69, this is a reduction of about 25 percent. Of the other innovation 
schools, Hillview and Summers each enjoyed reductions in the number of major disciplinary incidents 
per 9'-grade student as a result of PASL in at least one year, despite having increased total 
disciplinary incidents per student in both years. Among the five scale-out schools, only Meadows saw 
a reduction in disciplinary incidents attributable to the PASL intervention. The magnitude of the 
reduction was larger than that at Spring Gardens, -0.33 incidents per 9°"-grade student or about a 70 


percent reduction in the number of disciplinary incidents per student from the baseline level of 0.48. 


The schools with the most consistent evidence of a positive impact of PASL had the highest levels 
of implementation integrity. The effects on attendance and behavior were consistently positive and 
large for Spring Gardens. There is evidence that suggests the PASL experiment was associated with 
increases in attendance of 2.0 to 2.8 percentage points at one of the three innovation schools, Spring 
Gardens. This is quite substantial, given the average pre-intervention attendance rate of 90.5 percent 
at innovation schools. It is not clear that this boost was a direct result of PASL, however. The school- 
wide improvements in attendance during the second year of implementation at Spring Gardens 
mirrored those of 9" graders, despite the fact that only 9" graders participated in PASL in the second 
year. Neither of the other two innovation schools, Hillview and Summers, saw improvements in 
attendance as a result of PASL in the treated grades in either year. Similarly, only one of the five 
scale-out schools, Pleasant Valley, enjoyed a boost in attendance attributable to the intervention in 
2015-16. The gain for Pleasant Valley was smaller than that at Spring Gardens, a little over a one 


percentage point increase in attendance for 9""-grade students. 


Background 


Description of the Intervention 


The Broward County intervention departs from traditional research methods in its emphasis on 
scaling up effective programs and practices. In a traditional experimental approach, the primary 
emphasis is on obtaining unbiased estimates of the causal impact of an intervention. Typically a 
uniform intervention is applied to a relatively small (randomly selected) sample. In contrast, the 
process of intervention in Broward County incorporated concepts of improvement science and 
utilized a continuous improvement approach with the goal of reaching sustainable educational 
improvement. The continuous improvement methodology allows for cross-school variation in initial 
implementation, which is followed by rapid-cycle testing and evaluation of the intervention’s success. 
Subsequently, practitioners modify the intervention and the targeted sample of students to 


accommodate school-level conditions. 


To operationalize continuous improvement, district practitioners helped develop an intervention 
alongside researchers and development specialists known as Personalization for Academic and Social 
Emotional Learning (PASL). Broadly, PASL is a bundle of practices reflecting an increased awareness 
of the interdependency of academic and social-emotional activities in schools. PASL practices differ 


in intentional and unintentional ways across and within schools, but there are some common features 


to most PASL programs. Common practices include educator teams that collaborate to plan PASL 
activities, intentional points of contact such as routine and exploratory rapid check-ins or problem- 
solving meetings with students, norms of engagement such as high expectations or positive 
behavioral management, goal-achievement skill development, and use of data or other collected 
student information to inform subsequent PASL practices. Features of PASL that differed across 
schools include composition of the School Innovation Design Teams (SIDTs), selection process of PASL 
teachers, interpretation of PASL practices, collaboration between teachers on implementing PASL 
practices, school and teacher fidelity to newly-created PASL standards, the timing of different PASL 


components, and the frequency with which teachers exercised PASL practices. 


More abstractly, some schools compartmentalized PASL practices in a pattern of piecemeal adoption, 
while other schools integrated all PASL standards into a new school culture. Differences in PASL 
adoption across schools can be described as either mutual adaption of old practices with PASL or as 
cooption of existing practices with PASL terminology. In the case of cooption, few practices changed 
in a substantial way. Most differences in PASL adoption were caused by time and resource 
constraints, technical support, policy alignment with school missions, and teacher willingness to 


adopt the new practices. 


While the continuous improvement approach may hasten the adoption of an intervention and could 
yield better outcomes by quickly adapting the intervention to the specific context of a school, it also 
makes the task of evaluating the efficacy of the intervention very difficult. Since the intervention and 
its implementation are not uniform across schools, the effective sample may be reduced to a single 
school, limiting generalizability and statistical power to determine whether or not the intervention 
was effective. Likewise, to the extent that the intervention evolved over time, it is hard to know if 
changes in outcomes over time are due to fade-out of the efficacy of the intervention or to change in 


the nature of the intervention over time. 


Summaries of PASL Practices by School 


This section describes local PASL adaptations to educator teams, intentional points of contact (IPCs), 
norms of engagement, goal-achievement skill development, and intentional use of information. The 


section concludes with a matrix of PASL treatments by school, grade, and year. 


Spring Gardens High School — Innovation School 


Educator teams at Spring Gardens High School in the innovation year included all first period teachers, 


although monthly professional development and teacher cross-talks took place with a smaller subset 


of freshmen teachers. The second year of treatment scaled back to including just 9*"-grade instructors 
in the PASL program. Despite all instructors participating in PASL practices in the first year, the focus 
of implementation was largely on 9""-grade students. All 9*" graders were assigned a PASL instructor 
based on their 1* period class. Rapid check-ins (RCls) occurred daily in 1* period for most teachers, 
but different teachers engaged in RCls with varying frequency. Norms of engagement included PASL 
messaging, a school survey, a freshmen class assembly, and ice cream honor roll recognition. Goal 
achievement practices were administered during 1% period in the form of goal worksheets, 
extracurricular activity forms, weekly journaling of progress, and one written academic goal. 9" grade 
students also received planners. In addition, all teachers participated in a goal-writing professional 
development session. Intentional use of information came in the form of student support referrals 
and freshmen teacher cross-talks on focus students and “D & F” list students.? Spring Gardens High 
School was unique in administering several other adaptations. The school held quarterly school-wide 
cross talks, created a 9" grade mentoring program and crisis prevention support groups, 
supplemented 1* period lesson plans, and had decision making support lessons. Spring Gardens also 
made efforts to improve test scores by piloting a “Targeted Remediation Enrichment” of math and 


reading skills for 9° and 10" grade students. 


Summers High School — Innovation School 


Summers High School educator teams included 15 9'-grade HOPE (Health Opportunities with the 
Integration of Physical Education), English |, and Environmental Science teachers, although all 
teachers received three 30-minute professional development courses on creating a school culture. 
All 9*"-grade students were assigned to one of these teachers in groups of 18-25. Every three weeks 
these PASL educator teams engaged in rapid check-ins with their students, sending students with 
special needs to guidance counselors for additional support. Norms of engagement included daily 
PASL announcements, a PASL bulletin board, and a PASL goal-setting kiosk at lunch. PASL teachers 
provided goal achievement lessons to their students, and school assemblies were also used to 
advance goal achievement awareness. Intentional use of information involved quarterly discussion 
of student-level reports and goals sheets among PASL teachers of the same students, who would then 
collaborate to better serve the student’s needs. Summers also implemented English class book 


reports to be completed by students, which may have influenced reading test scores. 


1The precise nature of the “D & F” list is not entirely clear and may have varied across schools. In general, 
students on the D & F list were high-risk students with greater perceived needs. These lists were not static and the 
students on these lists could have changed during the year. 


Hillview High School — Innovation School 


Educator teams at Hillview included all 2"¢ period 9'"-grade teachers and teachers of HOPE classes. 
These teachers administered the PASL program to all 9'-grade students. Intentional points of contact 
were conducted twice per month as rapid check-ins before 2™ period and HOPE classes. Hillview also 
expanded intentional points of contact to include student mentors as well as administrator mentor 
groups for all students on the D & F list. Goal achievement lessons were taught in HOPE classes twice 
per year. The school shared the D & F list of high-needs students with teachers to make intentional 
use of information. Subsequent crosstalks occurred between teachers sharing the same D & F 
students. To make intentional use of information, students created quarterly goals sheets that were 
shared with all teachers using the Socrative application, a mobile student response system. Hillview’s 
unique adaptations include the “PASL pals” collaboration between school innovation design team 
members and small groups of PASL teachers. The school also made use of iObservation to reward 


teachers conducting rapid check-ins. 


Winter Ridge High School — Scale-Out School 
Winter Ridge High School assigned all 1% period 9'"-grade teachers to implement the PASL program 


for all 9'"-grade students. These teachers were instructed to administer intentional points of contact 
in the form of weekly rapid check-ins and follow-ups with students of concern. The school also 
increased contact by assigning upper-class students to meet with 9'"-grade students as peer mentors 
twice per month. Goal achievement lessons were administered through PASL teachers and the upper- 
class mentors. Counselors also met with focus students in need of additional support. The school 
made intentional use of information on student goals, student goal progress, and RCI frequency 
across PASL teachers. By sharing student goals with peer mentors, PASL faculty, school counselors, 
and administration, the school was able to assess student progress and identify students in need of 
additional support. School norms of engagement included mandatory 9*" grade club participation 
and teacher follow-ups with students asking questions. One unique feature of Winter Ridge’s PASL 
program was departmental lessons in studying skills. As part of these departmental lessons, math 
teachers taught organization, magnet teachers taught time management, English teachers taught 
decision making, science teachers taught stress management, and social studies teachers taught self- 


awareness. 


Pleasant Valley High School — Scale-Out School 
PASL teachers at Pleasant Valley High School were assigned based on location in any of four hallways, 
conditional on teaching a 9'"-grade 1%‘-period class. Reading and critical thinking teachers were 


excluded from PASL educator team assignment. Accordingly, approximately 535 students were 


assigned to a PASL teacher. Pleasant Valley provided intentional points of contact in the form of 
weekly rapid check-ins. Goal achievement lessons were taught in PASL classes. Guidance counselors 
and administrators held bi-monthly meetings with groups of ten students on the D & F list to provide 
follow-up goal achievement support. Pleasant Valley made use of information by creating a Google 
document of student goals and a D & F list of high-need students. The Google document and D & F 
list were shared with all teachers to facilitate teacher strategy crosstalks on students in common. 
Norms of engagement took the form of professional study days on the first Thursday of each month 
to discuss PASL data. 


White Mountain High School — Scale-Out School 


White Mountain High School assigned five teachers to implement the PASL program based on 
whether the instructors taught English or Math to 9'"-grade students in first or second period. As a 
result, only a select number of the 9' grade class (approximately 109 students) received PASL 
treatment. These five teachers participated in five professional development training sessions on 
PASL philosophy, PASL skills, goal achievement lessons, creating a school culture, and teacher 
collaboration. Intentional points of contact took the form of rapid check-ins during PASL classes every 
three weeks. Separately, guidance counselors held casual meetings with students of greatest need to 
provide additional support. Goal achievement lessons were delivered by English | teachers who 
collected goals sheets. Grade assemblies were also used to introduce goal achievement lessons. 
Intentional use of information took the form of quarterly specialist reports that helped to establish 
focal groups of students. Student goals sheets from English courses were shared with all teachers to 
facilitate crosstalks. White Mountain High School created new norms of engagement by changing the 


school mission statement to align with PASL and holding quarterly assemblies. 


Silver Oak High School — Scale-Out School 
Silver Oak High School assigned 14 teachers to implement PASL for about 150 9*" grade students, 


which means that only a subsample of all 9'" grade students received the treatment. PASL teachers 
conducted intentional points of contact in the form of rapid check-ins on a monthly basis at minimum. 
Goal achievement lessons took place during HOPE and JROTC classes. PASL teachers collected and 
weekly goal sheets and reinforced the goals. Although Silver Oak collected goals sheets, it is unclear 
whether these goals were shared among PASL teachers in the same way as other treatment schools 
made intentional use of information. D & F lists were shared among PASL teachers using a tailored 
information system. The school adopted norms of engagement through freshmen introductions, 


morning announcements, and adjustments to the school mission. 


Meadows High School — Scale-Out School 


Meadows High School assigned seven 4'"-period 9*"-grade teachers to implement PASL for the entire 
freshmen class of 179 students. Rapid check-ins were introduced in the fourth week of the semester. 
Guidance counselors taught goal achievement lessons during 4" period in the first week of the second 
quarter, and teachers supported goal achievement practices with follow-up assignments catering to 
class-specific curriculums. Meadows High School made intentional use of information in the form of 
D &F reports, which were distributed to PASL teachers sharing the same students. Teacher crosstalks 
on the goal reports took place on Thursday mornings. PASL norms of engagement included 
incorporation of all community members in PASL goals, promotion of high expectations, encouraging 
student participation in extracurricular activities, positive data-focused communication practices and 
alignment of the school mission to PASL. Meadows High School made a number of unique adaptations 
to the PASL program. In particular, the school delivered stress management lessons during 4" period 
that featured relevant GPA and graduation information. PASL teachers were also instructed to 


arrange “Wildcat Chats” with each other in addition to regular crosstalks. 


District Summary 


Figure 1 summarizes the timing and extent of the intervention across high schools and across grades 
within schools in Broward County. Cells that are blacked out indicate that all students within a given 
school and grade were targeted by the intervention, whereas a grayed-out cell indicates that only a 
subset of students within a school and grade were targeted. With the exception of Spring Gardens 


in the initial innovation year, only freshmen were targeted for the intervention. 


Figure 1: Broward County Treatments by School, Grade, and Year 
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Note: Cells shaded black indicate full exposure of the grade in that year; 
regions shaded gray indicate partial exposure of the grade in that school 
year. 


Data and Methods 


Data 


The Broward County data consists of individual-level information on all middle and high school 
students from the 2000-01 school year to the 2015-16 school year. The large size of the district means 
there are over 1 million student-year observations. Each observation includes time-constant 
demographic information such as race, gender, language spoken at home, disability status, and 
foreign-born status. Time-varying variables for each student include end-of-grade and end-of-course 
test scores, attendance rates, disciplinary incidents, free-and-reduced price lunch, limited English 
proficiency, gifted status, and number of schools attended in the school year. Information on class 


enrollment and class grades are also known but excluded from the analysis. Likewise, school-level 


demographics are known for some years, but missing data and changes to recording practices mean 


this information cannot be used for the regression analysis. 


Empirical Strategy 


To analyze the Broward County intervention in all its complexity, we utilize multiple empirical 
strategies over various analytic samples and across multiple student outcomes. We employ two types 
of analyses: a graphical analysis that compares trends over time between treated and not-treated 
schools and a multivariate regression analysis that controls for observable differences between 
students. Analytic samples in each school are all students, 9*"-grade students only, cohorts of 
individual students entering high school during one of the treatment years, and the combination of 
schools, years, and grade levels where at least some student received the PASL intervention. Student 
outcomes of interest over these analytic samples include attendance, total and major disciplinary 
incidents per student, and test scores in four subject areas. To convey the impact of PASL and 
continuous improvement practices across different school environments, all results are presented 


both by category of treatment and by school. 


Graphical assessment of the effects of the intervention is a short time-series design that compares 
student outcomes before and after the school-wide intervention. The essential feature of these 
graphs is the display of underlying pre-intervention trends at ever-treated and never-treated schools. 
Post-treatment departures from the pre-intervention period difference between ever-treated and 
never-treated schools provides suggestive evidence of the effects of the PASL treatment. Average 
trend lines, however, can be misleading because they do not account for changing demographics at 


schools or sample size differentials across treatment groups. 


To circumvent potential shortcoming of the graphical analysis, additional difference-in-difference 
regression strategies are employed across all outcomes and analytic samples. The estimates from 
these regressions show whether the pre/post treatment-period change in student outcomes is 
statistically different for treatment and control populations. As such, the regressions perform a 
similar exercise as the graphical analysis, but provide greater clarity by incorporating significance 
tests and controls for individual-level student characteristics. Three regression models are employed 


in the analysis, and within each model there are multiple specifications. 


Regression Model 1 — Difference-in-Differences 


The first regression model is a straightforward difference-in-differences regression. Intuitively, this 


regression specification asks whether the change in outcomes between the pre-treatment and post- 


treatment period for students at PASL schools differs from the change over the same time period for 
students who attend schools that never participated in PASL, conditional on observable student 
characteristics and district-wide trends. The simplest specification of this model, which corresponds 
analytically to the plotted figures by treatment, is of the form: 


Outcomeigs, = a + By Xiz + B2Year, + B3Gradejg, + ByEverPASLSchool;, (1.1) 
+ B;(EverPASLSchool;, X Post, ) + €igst 


In this equation, Xj, is a set of characteristics for student jin school s in year t. Additional controls for 
year and grade allow student average outcomes to differ across grades and years. The variable 
EverPASLSchool;, is an indicator for whether student i attends a school in year t that ever 
participates in PASL (i.e. an innovation or a scale-out school); it controls for average static differences 
between these schools and never treated schools. It is important to note that the indicator for schools 
that ever participate in PASL is an average across all innovation or scale-out schools, thus masking 
variation within the group of innovation and scale-out schools. The coefficient of interest is 6., which 
captures the effect of being in an innovation or a scale-out school during the years in which these 
schools received PASL treatment. For the innovation schools, the effect is estimated separately for 
the first and second year of treatment. In addition, equation (1.1) is estimated separately for the 
sample of 9*"-grade students (who were the only students receiving the PASL intervention, save for 
10-12" graders at Spring Gardens in the initial innovation year). By restricting the analysis sample 
to 9" graders, we limit treatment effects to those students actually being targeted, but we also ignore 


comparisons between treated and un-treated students within a school. 


A second specification of regression model (1) allows for the treatment effect to vary across ever- 


treated schools. This specification takes the form: 


Outcomejgs, = a + By Xiz + B2Year, + B3Gradejg, + BySchool;jsy (1.2) 
+ Bs(Schooljs, X Post; ) + Eigse 


where School;,, indicates that student i attends a particular ever-treated school s in year t. This 
second specification controls for the same time-varying student characteristics, year effects, and 
grade variables as does the first specification. In this specification, however, the scale-out and 
innovation indicators are replaced by individual school indicators. The vector 6; comprises the 
coefficients of interest in this specification, which are the estimated effects of being in a particular 
treated school in a year when the school received treatment. As before, the innovation school effect 


is estimated separately for the first and second year of treatment. 
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Regression Model 2 — Cohort Difference-in-Differences 


The second regression model employed is a cohort analysis. Whereas the analytic sample of the first 
regression model is all high school students, the cohort analysis restricts the sample to just 9"" grade 
students in the 2014-15 school year. The key difference between the initial and cohort-based analyses 
is that the identity of the students being studied is held constant and the grade level of students 
varies over time in the cohort analysis, whereas in the initial model the grade level is held constant 
and the identity of students varies over time. The cohort approach allows for two additional analyses. 
First, by holding the identity of students constant over time, one can determine if results from the 
initial analysis are influenced by changes in the composition of grade-level cohorts over time that 
happen to coincide with treatment. Second, the cohort approach allows one to determine if there 
are residual effects from participating in PASL. For example, 9" graders in innovation schools in 2014- 
15 participate in PASL, but they are no longer participating in PASL when they are observed as 10° 
graders in 2015-16. Performance gains in 2015-16 could therefore be attributed to a lasting effect of 
PASL participation in the prior year. The simplest specification of this model is of the form: 


Outcomeigse = a + Pi Xit + B2Gradejg, + B3EverPASLSchoolit (2.1) 
+ By(EverPASLSchool;, x Post, ) + €igst 


In this equation, controls for student characteristics and grade are the same as before. Year controls 
are dropped because the analytic sample is the same grade for all years, so the grade and year 
controls perform the same function. The indicator EverPASLSchool;, controls for static differences 
between cohorts that enter innovation schools in 2014-2015, allowing students that ever receive 
PASL to have different performance averages than untreated students. The coefficient of interest is 
B,, which captures the effect of entering an innovation school in 9" grade during PASL treatment 
years as opposed to entering a non-PASL school as a 9'"-grade student in the same year. The effect is 


estimated separately for the first and second year of treatment at innovation schools. 


A second specification of regression model (2) allows for the cohort treatment effect to vary across 


schools. This specification takes the form: 


Outcomejgs, = a + By Xit + BeGradejgz + B3Schoolis, + B4(Schoolis, X Post, ) + €igst (2.2) 


The second specification controls for the same time-varying student characteristics and grade 
variables. The scale-out and innovation indicators are replaced by individual school indicators. The 


vector B. comprises the coefficients of interest in this specification, which are the estimated effects 


11 


of being in a particular treated cohort in the year of treatment or the year after treatment. As before, 


the innovation school effect is estimated separately for the first and second year of treatment. 


Regression Model 3 — Difference-in-Differences-in-Differences 

The third regression model controls for dynamic variation across grades in different schools and 
years, allowing each grade-year, grade-school type and year-school type to have its own baseline 
average outcome. The first specification assumes zero persistence in treatment effects (see Figure 


1). The basic triple-difference regression model takes the following form: 


Outcomejgse = a + By, Xit + B2Year, + 63 Grade, + ByEverPASLSchoolijz (3.1.1) 
+ B;(EverPASLSchooli, x Gradéigt) 
+ Bg(EverPASLSchool;, x Year,) + By (Gradeigt x Year) 
+ BgPASL Cohortigt + €igst 


where X;; is a set of student characteristics for student iin year t, EverPASLSchool;, indicates that 
student /is enrolled in an ever-treated school s, Gradejg; are a set of dummies indicating that student 
iis enrolled in grade g in year t, and Year, are a set of effects for each year of the sample. The 
interaction term (EverPASLSchooli, x Gradeigr) allows for differing outcomes across ever- 
treated and never-treated schools at each grade level. The (EverPASLSchool;, x Year,) term 
allows for differing time trends between ever-treated and never-treated schools and (Gradeéjg_ x 
Year,) controls for differing time trends by grade level across all schools. PASL Cohort(,; represents 
a student who is enrolled in a school-grade-year combination in which either full or partial treatment 
occurred. The coefficient of interest in this specification is Bg, which is the effect of being in an ever- 
treated school in a year and grade in which the PASL intervention was carried out, with effects 


averaged across innovation and scale-out treatment groups. 


The difference between the difference-in-difference and the triple-difference approaches can be 
seen by comparing equation (3.1.1) to equation (1.1). The difference-in-difference model (1.1) does 
not include the two-way interactions, (EverPASLSchool, x Gradeigt), (EverPASLSchool;, X 
Year,), and (Grade; gt x Year;), and thus does not control for grade-specific and year-specific 
differences between ever-treated and never-treated schools. Most importantly, the triple-difference 
approach replaces the term (EverPASLSchool;, x Post, ) with (PASL Cohortjg,), meaning 
identification of the impact of the intervention comes from differences between treated and non- 


treated grades within a year in which a school received the PASL intervention. 


Equation (3.1.2) is a simple extension of the basic triple-difference specification that differentiates 


between full and partial treatment within a grade. Partial PASL;,, indicates a student was in a 


igt 
partially treated grade in year t. White Mountain and Silver Oak High School 9°" grade students in 
2015-2016 receive the partial indicator. Full PASL jg, indicates that a student was in a fully treated 


grade in a PASL school. 


Outcomejgst = & + B,Xix + B2Year, + BzGradejg, + ByEverPASLSchooliy (3.1.2) 
+ B;(EverPASLSchool;, x Gradejg,) 
+ Bg(EverPASLSchool;, x Year,) + B7(Gradejg, X Year,) 

+ BoPartial PASLigt + Py Full PASLigt + €igst 

A second specification of regression model (3) allows treatment effects to vary across schools. This 


specification takes the form: 


Outcome igs, = a + By Xit + B2Year, + B3Gradejg¢ + BySchooljct (3.2) 
+ Bs(Schoolise x Gradejgr) + B.(School;,, x Year; ) 
+ B;(Gradeigt x Year) + Ba(Schoolise x PASL Cohortigt ) + Eigse 


The second specification controls for the same time-varying student characteristics, year effects, 
grade variables, and interaction terms as equation (3.1.1). However, the scale-out and innovation 
indicators are replaced by individual school indicators. The vector 6g comprises the coefficients of 
interest in this specification, which are the estimated effects of being in a particular treated school in 


a year and grade when the school received treatment. 


Results 


Attendance 


Attendance measures the proportion of enrollment days over which a student is marked present, 
with no adjustment for excused absences. Broward County changed enrollment recording practices 
from 2012-13 to 2013-14, meaning that the observed attendance measurement after the 2012-13 
school year is a more precise accounting of attendance at each school a student attended in the year. 
For consistency, attendance measurements after 2012-13 are created by summing all regular- 


semester enrollments over the year and assigning a student to the school of longest enrollment. The 
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change in recording practices from 2012-13 to 2013-14 does not appear to have affected the figures 
or attendance regression analysis. It is important to note that averaging attendance masks 
idiosyncratic variation in attendance patterns across schools, grade levels, and student 
characteristics. Although averages are informative, there are distributional differences across schools 


that may not be captured by these figures and tables. 


Descriptive Evidence 


As illustrated in Figure 1, innovation schools tended to have lower attendance than never-treated 
schools prior to the innovation year, while scale-out schools tended to have higher attendance rates. 
In the innovation year, average attendance in high schools improved throughout the district, though 
the improvement appears to have been somewhat larger in the innovation schools. The scale-out 
schools, while also increasing attendance, did so less than the never-treated schools over the same 
period. Interestingly, both the schools that were treated in the innovation year (the innovation 
schools) as well as those that would not be treated until the following year (the scale-out schools) 
experienced comparable gains in attendance during the innovation year. The attendance patterns 
over time for high school freshman (Figure 2) mirror those for high schools as a whole. In contrast, if 
one looks only at the 2014-15 cohort of 9*"-grade students (Figure 3), there is little difference in 
attendance rates between innovation schools, scale-out schools and non-treated schools. Further, 
the attendance improvements experienced for 9" graders at Spring Gardens appear to be completely 


reversed when those students are in 10° grade. 


The summary statistics presented in Table 1 indicate that pre-treatment attendance patterns were 
roughly similar across innovation, scale-out, and never-treated high schools. Scale-out schools had 
the highest attendance levels, at nearly 92 percent, beating the innovation schools with 90.5 percent 
attendance and the never treated high schools with 91.3 percent. After treatment, attendance 
increased at all high schools in Broward County, but the largest gains were made by the innovation 
schools. These schools increased attendance by over four percentage points to a level mirroring the 
scale-out and never-treated schools. Scale-out schools increased average attendance from 91.93% to 
94.76%, or nearly three percentage points. The schools that never received PASL treatment increased 
attendance by three and a half percentage points, which is one percentage point less than the effect 


for scale-out schools and nearly one percentage point higher than the effect for innovation schools. 
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Figure 1. Attendance by Treatment Status 
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Figure 2. Attendance of 9" Graders by Treatment Status 
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Figure 3. Attendance by Treatment Status — 2014-15 9" Grade Cohort 
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Cohort includes only students not repeating or skipping a grade after entering middle school. 


Difference-in-Differences Results 


Table 2.1 presents results from the difference-in-difference estimation for both all grades at a school 
and for 9'"-grade students only. During the first and second years of treatment, both the innovation 
and the scale-out schools had comparable attendance changes relative to the never-treated (control) 


schools. 


Estimates from the difference-in-difference model that allows for variation in the effects of PASL 
among treated schools are presented in Table 2.2. Spring Gardens High School improved attendance 
relative to the control schools in the first and second years of treatment. Hillview and Summers High 
Schools had lower attendance relative to control schools in the first year of PASL, but were 
comparable in the second year of treatment. Among scale-out schools, Winter Ridge, Silver Oak, 
Meadows, and White Mountain High Schools saw reduced attendance relative to control schools, 


while Pleasant Valley High School had improved attendance. 


As shown in Table 3.1, among the cohort of students in 9" grade in 2014-15 there was no significant 
overall improvement in attendance at innovation schools (relative to the change in non-treated 
schools). Hillview was the only innovation school to see significantly different cohort attendance 


relative to the district, with attendance rates lowered by 2.0 percentage points during the year in 
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which the students received treatment, 2014-15 (Table 3.2). Interestingly, the effect of PASL 
participation appears to persist among the 2015-2016 Hillview cohort in their second year of 


treatment when they were no longer participating in the PASL treatment. 


Results from the triple-difference analysis, which controls for both changes over time in attendance 
rates between students in each grade in treated and non-treated schools and for school-wide changes 
over time for all students in treated schools, are presented in Tables 4.1 and 4.2. The findings are 
positive for Spring Gardens and Summers High Schools, negative for Hillview, Winter Ridge, and 
Meadows High Schools, and indistinguishable from zero for all other schools. The reason why these 
findings differ from the twice-differenced results can be seen by comparing the estimates in the first 
and second columns of Table 2.2. Take, for example, the case of Hillview. In 2014-15, the average 
impact for all grades was a decrease in the attendance rate of 1.5 percentage points and a decrease 
of 1.75 percentage points for 9'"-grade students. In the second year of treatment, the school-wide 
losses were indistinguishable from zero while 9" grade losses were 0.8 percentage points. The triple 
difference estimated effect of PASL on 9'"-grade attendance, at -0.92, is roughly the gap between the 
difference-in-difference estimated effects for students of all grades and for those in 9*"-grade. The 
triple difference estimates make this sort of within-school comparison and thus produce an estimated 


impact that is significantly different from zero. 


Table 1: Attendance by Treatment Status 


Innovation schools Scale-Out Schools Never Treated Schools 
Before After Before After Before 2014- After 2014- 
Treatment Treatment Treatment Treatment 2015 2015 
Attendance 90.47% 94.84% 91.93% 94.76% 91.3% 94.9% 
(0.100) (0.064) (0.090) (0.066) (0.0942) (0.0614) 
Observations 46,222 15,165 75,615 11,038 295,927 100,092 


Note: Displayed values are the average attendance of all students in all innovation, scale-out, and never-treated 
schools. Standard errors are in parentheses. 
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Table 2.1: Attendance Difference-in-Differences by Treatment 


Attendance Attendance 
Percentage - Percentage - 
All Grades Grade 9 
Innovation School x 2015 -0.158 -0.081 
(1.04) (1.05) 
Innovation School x 2016 0.589 0.446 
(0.55) (0.82) 
Scale-Out School x 2016 -0.592 -1.838 
(0.62) (0.77) 
Observations 1,093,079 317,128 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 
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Table 2.2: Attendance Difference-in-Differences by School 


Attendance Attendance 
Percentage - Percentage - 
All Grades Grade 9 
Spring Gardens x 2015 2.828 2.478 
(0.30) (0.33) 
Hillview x 2015 -1.483°"° -1.749""* 
(0.30) (0.33) 
Summers x 2015 -0.728" -0.151 
(0.30) (0.34) 
Spring Gardens x 2016 2.050°"° 2.363 
(0.28) (0.35) 
Hillview x 2016 0.337 -0.800"" 
(0.27) (0.34) 
Summers x 2016 -0.207 0.172 
(0.27) (0.34) 
Winter Ridge x 2016 -2.277""* -2.715°"* 
(0.24) (0.29) 
Silver Oak x 2016 -2.4338°"* -2.334"" 
(0.23) (0.26) 
Meadows x 2016 -0.539" -1.443°™* 
(0.25) (0.30) 
Pleasant Valley x 2016 1.218°° 1.375" 
(0.27) (0.34) 
White Mountain x 2016 -0.636°"* -0.818°"* 
(0.23) (0.28) 
Observations 1,093,079 317,128 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 
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Table 3.1: Cohort Attendance Difference-in-Differences by Treatment 


Cohort 
Attendance 
Percentage 
Innovation School x 2015 -0.219 
(0.57) 


Innovation School x 2016 -0.030 
(0.52) 
Observations 10,228 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for whether the school is an innovation or a scale-out school. Robust standard 
errors are clustered at the school level. 


Table 3.2: Cohort Attendance Difference-in-Differences by School 


Cohort 
Attendance 
Percentage 
Spring Gardens x 2015 1.179 
(1.03) 


Hillview x 2015 -1.953°" 
(0.42) 


Summers x 2015 -0.045 
(0.94) 


Spring Gardens x 2016 0.670 
(0.85) 


Hillview x 2016 -1.090™ 
(0.43) 


Summers x 2016 -0.267 
(0.79) 
Observations 10,228 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for whether the school is an innovation or a scale-out school. Robust standard 
errors are clustered at the school level. 
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Table 4.1: Attendance Difference-in-Differences-in-Differences by Treatment 


Attendance Attendance 
Percentage Percentage 
All Students in Treated 0.236 
Grades Receive PASL (0.51) 
Some Students in Treated -0.633 
Grades Receive PASL (0.42) 
Student Grade Receives 0.121 
Full or Partial PASL (0.51) 
Observations 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is ever treated. Interaction terms in the 
model include treatment-by-grade, treatment-by-year, and grade-by-year indicators. Robust standard errors are 
clustered at the school level. 
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Table 4.2: Attendance Difference-in-Differences-in-Differences by School 


Attendance 
Percentage 


Student Grade Receives 
PASL - Spring Gardens 


Student Grade Receives 
PASL - Hillview 


Student Grade Receives 
PASL - Summers 


Student Grade Receives 
PASL - Winter Ridge 


Student Grade Receives 
PASL - Silver Oak 


Student Grade Receives 
PASL - Meadows 


Student Grade Receives 
PASL - Pleasant Valley 


Student Grade Receives 
PASL - White Mountain 


0.744°"" 
(0.19) 


-0.922"™* 
(0.19) 


0.515°" 
(0.19) 


-0.702"™* 
(0.18) 


0.155 
(0.14) 

4443" 
(0.18) 


0.209 
(0.19) 


-0.218 
(0.15) 


Observations 


1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for each treated school. Interaction terms in the model include 
school-by-grade, school-by-year, and grade-by-year indicators. Robust standard errors are clustered at the school 


level. 
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Discipline 


Broward County records all suspensions, major disciplinary incidents, and minor disciplinary 
incidents. Major disciplinary incidents are serious crimes such as battery, possession of an illicit 
substance, vandalism, and arson. The measure of major disciplinary incidents per student in the 
following figures is the average number of major offenses committed by a given student in a school 
and year. Minor disciplinary incidents include lesser transgressions such as skipping class. The sum of 
major and minor incidents is total disciplinary incidents. Because the vast majority of total disciplinary 
incidents are minor incidents, analysis of minor disciplinary incidents is presented in the appendix. 
In-school suspensions per student and out-of-school suspensions are also reported in the appendix 


because they roughly correspond to the number of total and major disciplinary incidents. 


Descriptive Evidence 


Summary statistics for the number of disciplinary per student, broken down by school type and by 
pre/post treatment period are provided in Table 5. “Minor” incidents are about 10 times more likely 
than are major disciplinary incidents. Correspondingly, out-of-school suspensions are about 8 times 


more likely to occur than in-school suspensions. 


As illustrated in Figure 4, prior to the innovation year (2014-15), there was a general downward trend 
in the number of disciplinary incidents per student in high schools throughout the district. While 
innovation and never-treated schools exhibited similar levels of disciplinary incidents over time, the 
scale-out schools had substantially fewer disciplinary incidents per student in every year before the 
innovation year. In the innovation and scale-out years, the long-run downward trend flattened out, 
with the scale-out schools experiencing an uptick in disciplinary problems in the innovation year (the 


year prior to treatment for the scale-out schools). 


Major disciplinary incidents exhibit more variability from year to year than do the total number of 
incidents per student (Figure 5). This is not surprising given the relatively low number of major 
disciplinary incidents per student. As with the total number of incidents, the number of major 
infractions per student exhibited a general downward trend throughout Broward County high schools 


prior to the innovation year but remain roughly constant after that time. 


While there are notable changes from year to year, there are no reductions in disciplinary incidents 
per student that are clearly associated with PASL treatment. For example, while both total and major 
disciplinary incidents per student fall at Spring Gardens High School in the innovation year, these 


improvements in behavior appear to be part of a longer-term trend. Likewise, while total disciplinary 
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incidents per student rise at Summers High School during the innovation year, the upward trend 


appears to have started in the year prior to the introduction of PASL. 


The patterns for 9*" grade students (Figures 6 - 9), are similar to those for all students at a school. The 
only improvement that does not appear to be part of a longer-term trend is a drop in major 
disciplinary incidents per student at Hillview High School during the innovation year. This decrease 


does not continue for the second cohort of treated students in 2015-16, however. 


Figure 4. Total Number of Disciplinary Incidents per Student by Treatment Status 
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Figure 5. Number of Major Disciplinary Incidents per Student by Treatment Status 
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Figure 6. Total Number of Disciplinary Incidents per 9°"-Grade Student by Treatment Status 
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Figure 7. Number of Major Disciplinary Incidents per 9*"-Grade Student by Treatment Status 
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Figure 8. Total Number of Disciplinary Incidents by Treatment Status — 2014-15 9" Grade Cohort 
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2014-2015 9th Grade Cohort - Total Disciplinary Incidents 
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Cohort includes only students not repeating or skipping a grade after entering middle school. 


Figure 9. Number of Major Disciplinary Incidents by Treatment Status — 2014-15 9" Grade Cohort 
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Difference-in-Differences Results 


School-level per-student total disciplinary patterns did not significantly change in either the first or 
the second year of treatment for innovation schools as a whole, although average major disciplinary 
events decreased in the first year of treatment for innovation schools (Table 6.1). Similarly, as a group, 
the scale-out schools saw no changes to major or total disciplinary incidents. Among just 9""-grade 
students, innovation schools saw decreased major disciplinary incidents in both the 2014-15 and the 
2015-16 school year (relative to never-treated schools). With reductions in the number of major 
disciplinary incidents per student of -0.15 or -0.21, depending on year, this effect represents a 25 to 


40 percent decrease. 


There is evidence that Spring Gardens decreased major and total disciplinary incidents in the first and 
second year of treatment (Table 6.2). The impacts were substantial, ranging from a reduction of 0.14 
to 0.17 total incidents per student, depending on the grade level and year. Hillview High School 
managed to decrease major disciplinary incidents in the 2014-15 school year but also experienced an 
increase in the total number of incidents per student among 9" graders that year. Summers High 
School saw increases in total incidents in 2014-15 and 2015-16, but had decreases to major incidents 
in the second year of PASL treatment. Among scale-out schools, Meadows High School substantially 
decreased both major and total disciplinary incidents relative to never-treated schools in the 2015- 
16 school year. Meadows High School 9'"-grade students had both fewer major and total incidents 
than untreated 9'-grade students in the scale-out year, with decreases of roughly twice the 
magnitude as the effect at Spring Gardens in 2014-15 for total incidents. In contrast, Winter Ridge 
experienced increases in both major and total disciplinary incidents during the scale-out year, both 
for 9" graders and for the student body. Pleasant Valley saw increases to major and total disciplinary 
incidents in the scale-out year for 9""-grade students. Silver Oak achieved a slightly lower level of 
major disciplinary incidents without any change to total incidents for all students, but had increased 
total incidents for 9*"-grade students. White Mountain’s total disciplinary incidents went up in the 
first year of treatment for all students while major disciplinary incidents decreased among 9"-grade 


in the scale-out year. 


The cohort of 2014-15 freshman at innovation schools improved both major and total disciplinary 
incidents in the first year of treatment relative to 9*-grade students attending never-treated high 
schools in the same year (Table 7.1). The effect on total incidents does not persist into the 2015-16 
school year for total disciplinary incidents, although the effect remains negative but indistinguishable 


from zero for both major and total incidents. Spring Gardens High School appears responsible for 
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most of the innovation school effect, with its 9'"-grade cohort having one fewer disciplinary incident 
per student in both the 2014-15 and 2015-16 school years (Table 7.2). Since the average number of 
total disciplinary incidents per student for never-treated schools is about 0.7, this represents a fairly 
drastic change. Although effects are generally negative for the 9'"-grade cohort of Summers in both 
treatment years, the effects are not significantly different from zero at standard confidence levels. 
The cohort of 9'"-grade students at Hillview had significantly higher total disciplinary incidents per 
student in the 2015-16 year after treatment but had comparable rates of disciplinary incidents in the 
2014-15 PASL innovation year. 


Most, though not all, of the improvements in student behavior can be attributed to the PASL 
intervention when we employ the triple-difference approach, which makes comparisons across 
treated and untreated grades within ever-treated schools. Estimates from the triple-difference 
model, which are presented in Tables 8.1 and 8.2, indicate that the intervention was effective in 
lowering the number of major disciplinary incidents per student at Spring Gardens, Summers, 
Meadows and White Mountain. However, the reduction in total disciplinary incidents for Spring 
Gardens in the difference-in-difference analysis is no longer statistically different from zero in the 


tripe-difference analysis. 
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Table 5: Disciplinary Incidents per Student by Treatment 


Innovation Schools Scale-Out Schools Never Treated Schools 
Before After Before After Before After 
Treatment Treatment Treatment Treatment 2014-2015 2014-2015 
Major Incidents 0.0561 0.0353 0.0471 0.0378 0.052 0.0433 
(0.27) (0.21) (0.26) (0.24) (0.27) (0.24) 
Minor Incidents 0.6344 0.3791 0.4282 0.228 0.664 0.4476 
(1.78) (1.22) (1.34) (0.94) (1.88) (1.47) 
Total Incidents 0.6905 0.4144 0.4752 0.2658 0.716 0.4909 
(1.87) (1.28) (1.43) (1.02) (1.97) (1.55) 
Out-of-School 0.0527 0.0245 0.0591 0.0161 0.0726 0.0241 
Suspensions (0.30) (0.18) (0.33) (0.14) (0.38) (0.20) 
In-School 0.3915 0.191 0.1828 0.0883 0.2973 0.2181 
Suspensions (1.20) (0.75) (0.74) (0.45) (0.99) (0.84) 
Observations 46,222 15,165 76,350 11,038 295,927 100,092 


Note: Displayed values are the average number of disciplinary incidents per student across all students in treatment, 
treatment expansion, and never treated schools. Standard errors are in parentheses. 


Table 6.1: Disciplinary Event Difference-in-Differences by Treatment 


No. Total No. Total No. Major No. Major 
Discipline Discipline Discipline Discipline 
Incidents per Incidents per Incidents per Incidents per 
Student - All Student - 9th Student - All Student - 9th 
Grades Grade Grades Grade 
Innovation School x 2015 0.027 0.172 -0.015° -0.021° 
(0.08) (0.14) (0.01) (0.01) 
Innovation School x 2016 0.078 0.123 -0.003 -0.015"" 
(0.11) (0.15) (0.00) (0.01) 
Scale-Out School x 2016 -0.005 -0.037 0.001 0.005 
(0.10) (0.13) (0.01) (0.03) 
Observations 1,093,079 317,128 1,093,079 317,128 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 
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Table 6.2: Disciplinary Event Difference-in-Differences by School 


No. Total No. Total No. Major No. Major 
Discipline Discipline Discipline Discipline 
Incidents per Incidents per Incidents per Incidents per 
Student - All Student - 9th Student - All Student - 9th 
Grades Grade Grades Grade 
Spring Gardens x 2015 -0.137"° -0.168"° -0.030°" -0.040°"" 
(0.05) (0.07) (0.00) (0.01) 
Hillview x 2015 0.041 0.341" -0.017""" -0.025""" 
(0.05) (0.07) (0.00) (0.01) 
Summers x 2015 0.138" 0.257 -0.000 -0.001 
(0.05) (0.07) (0.00) (0.01) 
Spring Gardens x 2016 -0.154"" -0.172"° -0.012"" -0.020°"* 
(0.05) (0.07) (0.00) (0.01) 
Hillview x 2016 0.240°** 0.354°** -0.002 -0.008 
(0.05) (0.07) (0.00) (0.01) 
Summers x 2016 0.095° 0.161** 0.005 -0.018°"* 
(0.05) (0.07) (0.00) (0.01) 
Winter Ridge x 2016 0.131°" 0.149°* 0.018°"° 0.026 
(0.05) (0.06) (0.00) (0.01) 
Silver Oak x 2016 0.063 0.130" -0.007"" -0.009 
(0.05) (0.06) (0.00) (0.01) 
Meadows x 2016 -0.230°" -0.332"" -0.015°" -0.036" 
(0.05) (0.06) (0.00) (0.01) 
Pleasant Valley x 2016 0.070 0.137" 0.054" 0.160°"° 
(0.05) (0.06) (0.00) (0.01) 
White Mountain x 2016 0.153" 0.013 -0.002 -0.020°"* 
(0.05) (0.06) (0.00) (0.01) 
Observations 1,093,079 317,128 1,093,079 317,128 


Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 
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Table 7.1: Cohort Disciplinary Events Difference-in-Differences by Treatment 


(1) (2) 
No. Cohort No. Cohort 


Total Major 
Discipline Discipline 
Incidents Incidents 
per Student per Student 
Innovation School x 2015 -0.498° -0.059"" 
(0.26) (0.03) 
Innovation School x 2016 -0.379 -0.023 
(0.28) (0.03) 
Observations 10,228 10,228 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year and for whether the 
school is an innovation or a scale-out school. Robust standard errors are clustered at the school level. 
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Table 7.2: Cohort Disciplinary Events Difference-in-Differences by School 
No. Cohort No. Cohort 


Total Major 
Discipline Discipline 
Incidents Incidents 
per Student per Student 
Spring Gardens x 2015 -1.008""" -0.064 
(0.25) (0.04) 
Hillview x 2015 0.164 -0.030 
(0.16) (0.03) 
Summers x 2015 -0.629 -0.074° 
(0.47) (0.04) 
Spring Gardens x 2016 -0.968""" -0.069" 
(0.21) (0.03) 
Hillview x 2016 0.390°"* 0.017 
(0.15) (0.02) 
Summers x 2016 -0.570 -0.014 
(0.39) (0.03) 
Observations 10,228 10,228 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year and for whether the 
school is an innovation or a scale-out school. Robust standard errors are clustered at the school level. 
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Table 8.1: Disciplinary Event Difference-in-Differences-in-Differences by Treatment 


No. Total No. Total No. Major No. Major 
Discipline Discipline Discipline Discipline 
Incidents per Incidents per Incidents per Incidents per 
Student Student Student Student 
Schools Ever -0.262""" -0.262""" -0.003 -0.003 
Implementing PASL (0.08) (0.08) (0.01) (0.01) 
All Students in Treated -0.081 -0.017™ 
Grades Receive PASL (0.13) (0.01) 
Some Students in Treated -0.038 -0.027° 
Grades Receive PASL (0.06) (0.01) 
Student Grade Receives -0.076 -0.019"* 
Full or Partial PASL (0.12) (0.01) 
Observations 1,093,079 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is ever treated. Interaction terms in the 
model include treatment-by-grade, treatment-by-year, and grade-by-year indicators. Robust standard errors are 
clustered at the school level. 
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Table 8.2: Disciplinary Event Difference-in-Differences-in-Differences by Treatment 


No. Total No. Major 
Discipline Discipline 
Incidents per Incidents per 

Student Student 
Student Grade Receives 0.007 -0.010"° 
PASL - Spring Gardens (0.03) (0.00) 
Student Grade Receives 0.313°" -0.006 
PASL - Hillview (0.03) (0.00) 
Student Grade Receives 0.177" -0.010"° 
PASL - Summers (0.03) (0.00) 
Student Grade Receives 0.011 0.010" 
PASL - Winter Ridge (0.03) (0.00) 
Student Grade Receives 0.086" -0.002 
PASL - Silver Oak (0.04) (0.00) 
Student Grade Receives -0.126°"* -0.026""* 
PASL - Meadows (0.03) (0.00) 
Student Grade Receives 0.159" 0.145°"° 
PASL - Pleasant Valley (0.04) (0.00) 
Student Grade Receives -0.141°" -0.019""* 
PASL - White Mountain (0.04) (0.00) 
Observations 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is ever treated. Interaction terms in the 
model include treatment-by-grade, treatment-by-year, and grade-by-year indicators. Robust standard errors are 


clustered at the school level. 
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Test Scores 


The analysis of the effects of PASL on student achievement is restricted to test scores from the 
reading end-of-grade exam and the Algebra |, Biology, and Geometry end-of-course exams. Other 
end-of-grade or end-of-course exams were excluded because they either were not given to 9""-grade 
students at PASL schools during treatment years (e.g. exams in Science, Math, Civics, and American 
History) or they were only given after PASL treatment had occurred at the innovation schools (e.g. 
Algebra 2). Further, each of the end-of-course exams were administered to students at varying grade 
levels during the PASL intervention, so the estimated effects of the PASL intervention may be 
conflating the actual effects of PASL with the mix of students who take the exams at particular grade 
levels. The Appendix contains a section entitled, “School Patterns in End-of-Course Test Taking Over 
Time,” which provides a description of changes in the grade level at which students take end-of- 
course exams and an explanation of why these could lead to biased estimates of the achievement 
effects of PASL. All test scores are normalized by subtracting the average score within each grade 
from an individual student’s score and dividing by the standard deviation of scores across the district. 
Normalization of test scores allows easier comparison of performance across tests with differing 


score scales and achievement distributions. 


Descriptive Evidence 


The pattern of average test scores across treatment categories corresponds roughly to that found for 
attendance and for disciplinary incidents. Innovation schools have the lowest pre-treatment scores 
on all examined tests, while scale-out schools had the highest pre-treatment test scores (see Table 9 
and Figures 10, 13, 15 and 17). The plots of average test scores over time suggest that PASL did not 
lead to consistent improvement in student achievement. Average test scores declined in innovation 
schools during both the innovation and scale-out years in Reading. For the scale-out schools, reading 
scores dropped during the treatment year while never-treated schools experienced modest 
improvement in reading scores during the scale-out year. For Algebra |, Biology and Geometry, 
average test scores rose in the first year of PASL treatment at innovation schools, then fell back during 
the second (scale-out) year. Test performance in the scale-out schools varied considerably across 
subjects. The scale-out schools improved relative to never-treated schools in Algebra | and Biology, 
but fell in comparison to never-treated school in Geometry. Among innovation schools, the only 
school with test score gains in both the innovation and scale-out years was Summers, with gains in 
average biology test scores, relative to never-treated schools. Results are similar when limiting the 
sample to 9'" graders, as the majority of students taking the selected end-of-course exams are 
freshman (see Figures 11, 14, 16, 18). 
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Figure 10. Reading Test Scores by Treatment Status 
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Figure 11. Reading Test Scores of 9" Graders by Treatment Status 


9th Grade Reading Test Scores by Treatment 
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Figure 12. Reading Test Scores by Treatment Status — 2014-15 9'* Grade Cohort 


2014-2015 9th Grade Cohort - Reading Test Scores 
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Cohort includes only students not repeating or skipping a grade after entering middle school. 


Figure 13. Algebra | Test Scores by Treatment Status 
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Figure 14. Algebra | Test Scores of 9*" Graders by Treatment Status 
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Figure 15. Biology Test Scores by Treatment Status 
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Figure 16. Biology Test Scores of 9'" Graders by Treatment Status 
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Figure 17. Geometry Test Scores by Treatment Status 
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Figure 18. Geometry Test Scores of 9*° Graders by Treatment Status 


9th Grade Geometry Test Scores by Treatment 
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Difference-in-Differences Results 


Estimated impacts of the PASL intervention for innovation schools as a group and scale-out schools 
as a group are provided in Tables 10.1 (for all students) and 10.2 (for only 9" graders). Since the PASL 
intervention primarily targeted 9" graders, we focus on the 9'-grade only results. Collectively, 
innovation schools saw no significant changes to any test score outcome in either year of PASL 
treatment. Similarly, there were also no significant impacts on test scores in any of the four subjects 


for scale-out schools during their first year of implementation. 


Average test score changes mask considerable variation among PASL innovation schools. 9'"-grade 
students at Spring Gardens saw improved Biology scores in both years of treatment offset by declines 
in reading scores (see Tables 10.3 and 10.4). Improvements to Algebra | and Geometry for Spring 
Gardens increased in the first year and then decreased in the second year of PASL implementation. 
During the innovation year, Hillview improved Algebra | scores but lost ground in Biology; in the 
second year of PASL, however, Hillview increased Reading, Algebra |, and Geometry test scores 
relative to the comparison control. Summers saw large improvements to Geometry and Biology end- 
of-course exams in the innovation year despite reductions in reading scores. During the scale-out 


year, Summers High School reversed its reading test score results from the innovation year and 
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experienced persistently positive Geometry and Biology test scores. Summers’s gains in Biology may 


stem from their selection of Biology and Earth Science teachers for the PASL intervention. 


Among the scale-out schools, Winter Ridge and Pleasant Valley experienced relative improvement in 
reading scores in 2015-16 of 0.10 and 0.06 standard deviations, respectively. In contrast, White 
Mountain and Silver Oak each had decreased relative reading scores for 9*"-grade students. The 
scale-out schools were split between gains in Algebra | scores at Meadows and Pleasant Valley and 
decreases at Silver Oak and White Mountain. Pleasant Valley improved test scores while White 
Mountain experienced a statistically significant decrease in Geometry scores for 9"" graders as a result 
of PASL. In Biology, Pleasant Valley was the only one of the five scale-out schools to obtain relative 


test score improvements for their 9" graders. 


When the analysis is limited to the cohort of students who are high school freshman in 2014-15, we 
find no significant positive effects of PASL on student reading scores. This is true both for the 
innovation schools as a whole (Table 11.1), although Hillview appears to have improved reading 
scores among the cohort of 2014-15 9"-grade students in the scale-out year (Table 11.2).? In one of 
the innovation schools, Spring Gardens, the initial year of PASL treatment is associated with declines 


in relative reading scores, and these negative effects persist in the second year of implementation. 


The estimated impacts of PASL on test scores are mixed when employing the triple difference model, 
which compares students across treated and untreated grades within schools. In no case were there 
statistically significant gains in relative test scores attributable to the PASL intervention for innovation 
schools as a group or for scale-out schools as a group (see Tables 12.1 and 12.2). For reading, the 
estimated effects of PASL participation were consistently negative, ranging from -0.03 to -0.11 
standard deviations, depending on the specification. Furthermore, the negative effects on reading 
scores were found for all but three of the eight schools where PASL was implemented. There were, 
however, some instances where specific schools experienced test score gains on particular end-of- 
course exams (see Table 12.3). These included Hillview, Winter Ridge and Silver Oak in Algebra |; 
Hillview, Summers and Pleasant Valley in Biology; and Summers, Winter Ridge, Silver Oak and 
Coconut Beach in Geometry. These results must be viewed with caution, however, since they are 
based on cross-grade comparisons and students who take the same end-of-course exam at different 


grade levels may differ in unobservable ways that are correlated with test performance. 


Table 9: Test Scores by Treatment Status 


2 Note that since students only take end-of-course exams once, the single-cohort analysis (which relies on multiple 
observations of the outcome variable for a student over time) is not relevant. 
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Innovation Schools Scale-Out Schools Never Treated Schools 
Before After Before After Before After 
Treatment Treatment Treatment Treatment 2014-2015 2014-2015 
Reading -0.116 -0.223 0.157 0.157 0.007 -0.008 
(0.95) (0.98) (0.94) (0.98) (1.00) (1.00) 
Observations 27,332 7,666 41,808 6,153 196,783 61,073 
Algebra 1 -0.379 -0.403 -0.154 -0.148 -0.292 -0.381 
(0.86) (0.85) (0.87) (0.82) (0.92) 0.96) 
Observations 6,153 2,831 9,858 1,703 42,999 22,384 
Biology -0.188 -0.328 0.179 0.199 -0.022 -0.090 
(0.94) (0.93) (0.94) (0.90) (1.01) (1.01) 
Observations 4,951 3,387 10,243 2,455 37,881 27,445 
Geometry -0.286 -0.316 0.078 -0.012 -0.144 -0.153 
(0.89) (0.85) (0.83) (0.86) (0.95) (0.98) 
Observations 4,321 3,303 8,563 2,355 32,879 24,800 


Note: Displayed values are the average number of disciplinary incidents per student across all students in innovation, 
scale-out, and never treated schools. Standard errors are in parentheses. 


Table 10.1: Test Score Difference-in-Differences for All Grades by Treatment 


Reading - Algebra1- Geometry - Biology - 
All Grades All Grades All Grades All Grades 


Innovation School x 2015 -0.017 0.048 0.150" 0.049 
(0.02) (0.05) (0.07) (0.06) 
Innovation School x 2016 0.005 -0.021 0.030 0.012 
(0.03) (0.09) (0.03) (0.08) 
Scale-Out School x 2016 0.019 0.050 -0.049 -0.073 
(0.02) (0.08) (0.03) (0.05) 
Observations 471,234 62,006 55,610 63,782 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year, for the proportion of 
students taking a given End-of-Course exam before high school, and for whether the school is an innovation or a 
scale-out school. Robust standard errors are clustered at the school level. 


Table 10.2: Test Score Difference-in-Differences for 9th Grade by Treatment 


Reading - Algebra1- Geometry - Biology - 
9th Grade 9th Grade 9th Grade 9th Grade 
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Innovation School x 2015 -0.046 0.073 0.161 0.022 
(0.03) (0.05) (0.10) (0.07) 
Innovation School x 2016 0.001 0.010 0.080 0.082 
(0.03) (0.08) (0.05) (0.08) 
Scale-Out School x 2016 -0.016 0.038 -0.033 -0.069 
(0.03) (0.09) (0.05) (0.05) 
Observations 232,090 52,910 13,007 29,664 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year, for the proportion of 
students taking a given End-of-Course exam before high school, and for whether the school is an innovation or a 
scale-out school. Robust standard errors are clustered at the school level. 


Table 10.3: Test Score Difference-in-Differences for All Grades by School 


Reading - Algebra1- Geometry - Biology - 
All Grades All Grades All Grades All Grades 


Spring Gardens x 2015 -0.034"* 0.079" 0.289°"* 0.141°™* 
(0.02) (0.03) (0.03) (0.02) 
Hillview x 2015 0.019 0.099" 0.054 -0.070°" 
(0.02) (0.04) (0.04) (0.02) 
Summers x 2015 -0.040" -0.023 0.151°" 0.116°" 
(0.02) (0.04) (0.04) (0.01) 
Spring Gardens x 2016 -0.018" -0.158°" -0.046" 0.056 
(0.01) (0.03) (0.02) (0.04) 
Hillview x 2016 0.060" 0.149°** 0.075" -0.124"" 
(0.01) (0.03) (0.03) (0.03) 
Summers x 2016 -0.030"** -0.110°** 0.063 0.143*** 
(0.01) (0.03) (0.04) (0.02) 
Winter Ridge x 2016 0.089*** -0.083 -0.231°"* -0.080*** 
(0.01) (0.08) (0.07) (0.03) 
Silver Oak x 2016 -0.050°" -0.120°" -0.034 -0.069" 
(0.01) (0.02) (0.02) (0.03) 
Meadows x 2016 0.021" 0.188" -0.041° -0.171°"" 
(0.01) (0.03) (0.02) (0.03) 
Pleasant Valley x 2016 0.079°"* 0.142°"* -0.094"* 0.041" 
(0.01) (0.02) (0.04) (0.02) 
White Mountain x 2016 -0.026"" 0.006 0.033 -0.020 
(0.01) (0.03) (0.03) (0.03) 
Observations 471,234 62,006 55,610 63,782 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year, for the proportion of 
students taking a given End-of-Course exam before high school, and for whether the school is an innovation or a 
scale-out school. Robust standard errors are clustered at the school level. 
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Table 10.4: Test Score Difference-in-Differences for 9th Grade by School 


Reading - Algebra1- Geometry - Biology - 
9th Grade 9th Grade 9th Grade 9th Grade 


Spring Gardens x 2015 -0.075°"* 0.147°** 0.350°" 0.050" 
(0.02) (0.04) (0.03) (0.02) 
Hillview x 2015 0.005 0.084" 0.022 -0.086°"" 
(0.02) (0.04) (0.04) (0.02) 
Summers x 2015 -0.082*** 0.002 0.169" 0.189°** 
(0.02) (0.04) (0.05) (0.02) 
Spring Gardens x 2016 -0.052""" -0.144"" -0.077" 0.095" 
(0.01) (0.03) (0.03) (0.05) 
Hillview x 2016 0.060°** 0.150" 0.132°" -0.039 
(0.01) (0.03) (0.04) (0.03) 
Summers x 2016 -0.003 -0.052 0.179"** 0.239*** 
(0.01) (0.03) (0.05) (0.03) 
Winter Ridge x 2016 0.095*** -0.079 -0.204"* -0.022 
(0.01) (0.09) (0.08) (0.03) 
Silver Oak x 2016 -0.070°"* -0.101°™* 0.034 -0.088°** 
(0.01) (0.02) (0.03) (0.03) 
Meadows x 2016 -0.021 0.183" -0.004 -0.156°" 
(0.01) (0.03) (0.03) (0.03) 
Pleasant Valley x 2016 0.056 0.122°"* 0.169°"° 0.215" 
(0.01) (0.03) (0.04) (0.03) 
White Mountain x 2016 -0.096"** -0.116°* -0.112"" -0.029 
(0.01) (0.03) (0.03) (0.03) 
Observations 232,090 52,910 13,007 29,664 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year, for the proportion of 
students taking a given End-of-Course exam before high school, and for whether the school is an innovation or a 
scale-out school. Robust standard errors are clustered at the school level. 
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Table 11.1: Cohort Reading Test Difference-in-Differences by Treatment 


Cohort 

Reading 
Innovation School x 2015 -0.039 

(0.09) 


Innovation School x 2016 0.002 
(0.07) 
Observations 5,015 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for whether the school is an 
innovation or a scale-out school. Robust standard errors are clustered at the school level. 


Table 11.2: Cohort Reading Test Difference-in-Differences by School 


Cohort 
Reading 
Spring Gardens x 2015 -0.169°"* 
(0.06) 
Hillview x 2015 0.045 
(0.04) 
Summers x 2015 -0.056 
(0.05) 
Spring Gardens x 2016 -0.081° 
(0.05) 
Hillview x 2016 0.074" 
(0.03) 
Summers x 2016 -0.048 
(0.04) 
Observations 5,015 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
8th grade reading and math scores, individual indicators for gender, foreign-born status, primary language spoken 
at home, race, disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for whether the school is an 
innovation or a scale-out school. Robust standard errors are clustered at the school level. 
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Table 12.1: Test Score Difference-in-Differences-in-Differences by Treatment 


Reading Algebra 1 Biology Geometry 
Schools Ever -0.023 -0.050 0.047 -0.042 
Implementing PASL (0.02) (0.06) (0.05) (0.06) 
Student Grade Receives -0.083°"" 0.019 -0.006 0.028 
Full or Partial PASL (0.02) (0.04) (0.04) (0.03) 
Observations 471,234 62,006 63,782 55,610 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for each treated school. Interaction terms in the model include 
school-by-grade, school-by-year, and grade-by-year indicators. Robust standard errors are clustered at the school 
level. 


Table 12.2: Test Score Difference-in-Differences-in-Differences by Treatment with Differential Effects 
for Partial Exposure 


Reading Algebra 1 Biology Geometry 
Schools Ever -0.023 -0.050 0.046 -0.043 
Implementing PASL (0.02) (0.06) (0.05) (0.06) 
All Students in Treated -0.079""" 0.027 -0.023 0.018 
Grades Receive PASL (0.03) (0.04) (0.05) (0.03) 
Some Students in Treated -0.106""" -0.141" 0.105 0.091 
Grades Receive PASL (0.03) (0.07) (0.07) (0.10) 
Observations 471,234 62,006 63,782 55,610 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is ever treated. Interaction terms in the 
model include treatment-by-grade, treatment-by-year, and grade-by-year indicators. Robust standard errors are 
clustered at the school level. 
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Table 12.3: Test Score Difference-in-Differences-in-Differences by School 


Reading Algebra 1 Biology Geometry 
Student Grade Receives -0.060°"* -0.310°"* 0.026 -0.102*** 
PASL - Spring Gardens (0.01) (0.04) (0.03) (0.03) 
Student Grade Receives -0.005 0.105° 0.085°"* -0.012 
PASL - Hillview (0.01) (0.05) (0.02) (0.03) 
Student Grade Receives -0.014 -0.016 0.138" 0.120" 
PASL - Summers (0.01) (0.10) (0.02) (0.03) 
Student Grade Receives 0.001 0.457°" -0.028 0.095" 
PASL - Winter Ridge (0.01) (0.05) (0.04) (0.04) 
Student Grade Receives -0.044°"* 0.675" -0.015 0.090" 
PASL - Silver Oak (0.01) (0.04) (0.03) (0.04) 
Student Grade Receives -0.106°"* -0.167°"* 0.047 -0.043 
PASL - Meadows (0.01) (0.04) (0.03) (0.03) 
Student Grade Receives -0.026"" 0.078 0.328°"* 0.259°"* 
PASL - Pleasant Valley (0.01) (0.05) (0.03) (0.03) 
Student Grade Receives -0.143°" -0.092° 0.036 -0.183°"* 
PASL - White Mountain (0.01) (0.05) (0.03) (0.03) 
Observations 471,234 62,006 63,782 55,610 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 


free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for each treated school. Interaction terms in the model include 
school-by-grade, school-by-year, and grade-by-year indicators. Robust standard errors are clustered at the school 
level. 
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Appendix 


Selection of Comparison Control Schools 


All figures, summary statistics, and regression output make use of a comparison group of 22 never- 
treated schools in Broward County. These schools were selected from a group of 74 other schools 
serving high school students because of their similarity in student population and institutional factors 
to the innovation and scale-out schools. The comparison group excludes technical schools, charter 
schools, magnet schools, alternative schools for students with disabilities or behavioral problems and 


continuation schools serving non-high school grades. 


Table A.1: List of Innovation, Scale-Out, and Control Schools 


Innovation Schools Scale-Out Schools Never Treated Schools 
Spring Gardens HS Pleasant Valley HS All other (22) non-alternative 
Hillview HS Winter Ridge HS high schools 
Summers HS Silver Oak HS 
Meadows HS 
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Minor Disciplinary Incidents 


Figure A.1: Number of Minor Disciplinary Incidents per Student by Treatment Status 
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Figure A.2: Number of Minor Disciplinary Incidents per 9*"-Grade Student by Treatment Status 
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Figure A.3: Number of Minor Disciplinary Incidents by Treatment Status — 2014-15 9'" Grade 
Cohort 
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In-School Suspensions 


Figure A.4: Number of In-School Suspensions per Student by Treatment Status 
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Figure A.5: Number of In-School Suspensions per 9*"-Grade Student by Treatment Status 
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Figure A.6: Number of In-School Suspensions by Treatment Status — 2014-15 9" Grade Cohort 
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Out-of-School Suspensions 


Figure A.7: Number of Out-of-School Suspensions per Student by Treatment Status 
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Figure A.8: Number of Out-of-School Suspensions per 9*"-Grade Student by Treatment Status 
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Figure A.9: Number of Out-of-School Suspensions by Treatment Status — 2014-15 9" Grade Cohort 
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Cohort includes only students not repeating or skipping a grade after entering middle school. 
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All Disciplinary Events 


Table A.2: Disciplinary Event Difference-in-Differences for All Grades by Treatment 


No. Minor No. In-School No. Out-of- 


Discipline Suspensions School 
Incidents per per Student - Suspensions 
Student - All All Grades per Student - 
Grades All Grades 
Innovation School x 2015 0.041 -0.017 0.019 
(0.08) (0.04) (0.01) 
Innovation School x 2016 0.080 -0.003 0.025° 
(0.10) (0.06) (0.01) 
Scale-Out School x 2016 -0.007 -0.016 0.001 
(0.10) (0.04) (0.02) 
Observations 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 
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Table A.3: Disciplinary Event Difference-in-Differences for 9th Grade by Treatment 


(1) 


(2) 


(3) 


No. Minor No. In-School No. Out-of- 
Discipline Suspensions School 
Incidents per per Student - Suspensions 
Student - 9th 9th Grade per Student - 
Grade 9th Grade 
Innovation School x 2015 0.194 0.056 0.033" 
(0.14) (0.08) (0.02) 
Innovation School x 2016 0.138 0.021 0.032" 
(0.15) (0.07) (0.02) 
Scale-Out School x 2016 -0.042 -0.032 0.003 
(0.11) (0.05) (0.03) 
Observations 317,128 317,128 317,128 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 


Table A.4: Disciplinary Event Difference-in-Differences for All Grades by School 


(1) 


(2) 


(3) 


No. Minor No. In-School No. Out-of- 
Discipline Suspensions School 
Incidents per per Student - Suspensions 
Student - All All Grades per Student - 
Grades All Grades 
Spring Gardens x 2015 -0.107"° -0.096""* 0.024°** 
(0.05) (0.03) (0.01) 
Hillview x 2015 0.058 0.004 0.035°"" 
(0.05) (0.03) (0.01) 
Summers x 2015 0.138" 0.023 -0.006 
(0.05) (0.02) (0.01) 
Spring Gardens x 2016 -0.141"" -0.120°" 0.042°** 
(0.05) (0.03) (0.01) 
Hillview x 2016 0.242°"* 0.083°"* 0.035°"° 
(0.05) (0.03) (0.01) 
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Summers x 2016 0.090° 0.001 -0.000 


(0.05) (0.03) (0.01) 
Winter Ridge x 2016 0.113" 0.048" 0.039°"* 
(0.05) (0.03) (0.01) 
Silver Oak x 2016 0.070 0.059" 0.006 
(0.05) (0.03) (0.01) 
Meadows x 2016 -0.215°" -0.088°** 0.011" 
(0.05) (0.03) (0.01) 
Pleasant Valley x 2016 0.016 -0.064" -0.069""* 
(0.05) (0.03) (0.01) 
White Mountain x 2016 0.154" 0.006 0.026" 
(0.05) (0.02) (0.01) 
Observations 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 


Table A.5: Disciplinary Event Difference-in-Differences for 9th Grade by School 


(1) (2) (3) 
No. Minor No. In-School No. Out-of- 


Discipline Suspensions School 
Incidents per per Student - Suspensions 
Student - 9th 9th Grade per Student - 
Grade 9th Grade 
Spring Gardens x 2015 -0.128° -0.129""* 0.042*** 
(0.07) (0.04) (0.01) 
Hillview x 2015 0.366 0.131" 0.051°"" 
(0.07) (0.04) (0.01) 
Summers x 2015 0.258°"° 0.124°"* -0.002 
(0.07) (0.04) (0.01) 
Spring Gardens x 2016 -0.152"° -0.119"" 0.048°** 
(0.07) (0.04) (0.01) 
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ok 2K 2K 


Hillview x 2016 0.362 0.128 0.052 
(0.07) (0.04) (0.01) 
Summers x 2016 0.179°** 0.041 -0.004 
(0.06) (0.04) (0.01) 
Winter Ridge x 2016 0127" 0.062" 0.087°°° 
(0.06) (0.03) (0.01) 
Silver Oak x 2016 0.139" 0.088°"* 0.021" 
(0.06) (0.03) (0.01) 
Meadows x 2016 -0.296°"" -0.136°"" 0.021" 
(0.06) (0.03) (0.01) 
Pleasant Valley x 2016 -0.023 -0.088" -0.129""* 
(0.06) (0.04) (0.01) 
White Mountain x 2016 0.033 -0.048 0.018" 
(0.06) (0.03) (0.01) 
Observations 317,128 317,128 317,128 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is an innovation or a scale-out school. 
Robust standard errors are clustered at the school level. 


Table A.6: Cohort Disciplinary Event Difference-in-Differences by Treatment 


(1) (2) (3) 


No. Cohort No. Cohort In- No. Cohort 
Minor School Out-of-School 
Discipline Suspensions Suspensions 
Incidents per per Student per Student 
Student 
Innovation School x 2015 -0.440° -0.304™ -0.079" 
(0.25) (0.14) (0.04) 
Innovation School x 2016 -0.356 -0.273° -0.055 
(0.27) (0.14) (0.04) 
Observations 10,228 10,228 10,228 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born status, primary language spoken at home, race, disability by type, 
grade level, free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the 
school year. Additional controls include indicators for year and for whether the school is an innovation or a scale- 
out school. Robust standard errors are clustered at the school level. 
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Table A.7: Cohort Disciplinary Event Difference-in-Differences by School 


(1) (2) (3) 
No. Cohort No. Cohort No. Cohort 


Minor In-School Out-of- 
Discipline | Suspensions School 
Incidents per Student Suspensions 
per Student per Student 
Spring Gardens x 2015 -0.944""* -0.716°"° -0.060° 
(0.23) (0.16) (0.04) 
Hillview x 2015 0.194 0.101 -0.054 
(0.16) (0.10) (0.08) 
Summers x 2015 -0.555 -0.288 -0.129 
(0.45) (0.24) (0.08) 
Spring Gardens x 2016 -0.899°"* -0.679°"* -0.029 
(0.19) (0.13) (0.03) 
Hillview x 2016 0.374" 0.139 -0.038 
(0.15) (0.10) (0.07) 
Summers x 2016 -0.556 -0.275 -0.101 
(0.38) (0.20) (0.07) 
Observations 10,228 10,228 10,228 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born status, primary language spoken at home, race, disability by type, 
grade level, free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the 
school year. Additional controls include indicators for year and for whether the school is an innovation or a scale- 
out school. Robust standard errors are clustered at the school level. 
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Table A.8: Disciplinary Event Difference-in-Differences-in-Differences by Treatment 


(1) (2) (3) 


No. Minor No. In-School No. Out-of- 
Discipline Suspensions School 
Incidents per per Student Suspensions 
Student per Student 
Schools Ever -0.259""" -0.084" -0.046™ 
Implementing PASL (0.07) (0.05) (0.02) 
Student Grade Receives -0.057 -0.020 -0.012 
Full or Partial PASL (0.11) (0.06) (0.01) 
Observations 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. 
Controls include individual indicators for gender, foreign-born, primary language spoken at home, race, 
disability by type, grade level, free-and-reduced price lunch, limited English proficiency, gifted, and 
number of schools attended in the school year. Additional controls include indicators for year and for 
each treated school. Interaction terms in the model include school-by-grade, school-by-year, and grade- 
by-year indicators. Robust standard errors are clustered at the school level. 


Table A.9: Disciplinary Event Difference-in-Differences-in-Differences by Treatment 


(1) (2) (3) 
No. Minor No. In-School No. Out-of- 


Discipline Suspensions School 
Incidents per per Student Suspensions 

Student per Student 
Schools Ever -0.259°"" -0.084° -0.046" 
Implementing PASL (0.07) (0.05) (0.02) 
All Students in Treated -0.064 -0.023 -0.016 
Grades Receive PASL (0.13) (0.07) (0.01) 
Some Students in Treated -0.011 0.002 0.013 
Grades Receive PASL (0.06) (0.05) (0.01) 
Observations 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for whether the school is ever treated. Interaction terms in the 
model include treatment-by-grade, treatment-by-year, and grade-by-year indicators. Robust standard errors are 
clustered at the school level 
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Table A.10: Disciplinary Event Difference-in-Differences-in-Differences by Treatment 


(1) (2) (3) 
No. Minor No. In-School No. Out-of- 
Discipline Suspensions School 
Incidents per per Student Suspensions 
Student per Student 
Schools Ever -0.259""" -0.084" -0.046™ 
Implementing PASL (0.07) (0.05) (0.02) 
Student Grade Receives -0.057 -0.020 -0.012 
Full or Partial PASL (0.11) (0.06) (0.01) 
Observations 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for each treated school. Interaction terms in the model include 
school-by-grade, school-by-year, and grade-by-year indicators. Robust standard errors are clustered at the school 


level. 
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Table A.11: Disciplinary Event Difference-in-Differences-in-Differences by Treatment 


(1) (2) (3) 
No. Minor No. In-School No. Out-of- 


Discipline Suspensions School 
Incidents per per Student Suspensions 

Student per Student 
Student Grade Receives 0.017 0.009 0.011" 
PASL - Spring Gardens (0.03) (0.02) (0.01) 
Student Grade Receives 0.319" 0.120°°° 0.028°"* 
PASL - Hillview (0.03) (0.02) (0.01) 
Student Grade Receives 0.187°"° O15" 0.011° 
PASL - Summers (0.03) (0.02) (0.01) 
Student Grade Receives 0.000 0.023 0.059""* 
PASL - Winter Ridge (0.03) (0.02) (0.01) 
Student Grade Receives 0.088" 0.042* 0.017*** 
PASL - Silver Oak (0.04) (0.02) (0.01) 
Student Grade Receives -0.100°™* -0.059°™* 0.013" 
PASL - Meadows (0.03) (0.02) (0.01) 
Student Grade Receives 0.014 -0.006 -0.072""* 
PASL - Pleasant Valley (0.03) (0.02) (0.01) 
Student Grade Receives -0.121"" -0.070°" -0.001 
PASL - White Mountain (0.03) (0.02) (0.01) 
Observations 1,093,079 1,093,079 1,093,079 


Note: Standard errors are in parentheses. Results significant at the *** 1%, ** 5%, and * 10% level. Controls include 
individual indicators for gender, foreign-born, primary language spoken at home, race, disability by type, grade level, 
free-and-reduced price lunch, limited English proficiency, gifted, and number of schools attended in the school year. 
Additional controls include indicators for year and for each treated school. Interaction terms in the model include 
school-by-grade, school-by-year, and grade-by-year indicators. Robust standard errors are clustered at the school 
level. 
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Table A.12: Percentage of 9*" Grade End-of-Course Test Takers 


Algebra | Biology Geometry 
School Year # Test Percentage in # Test Percentage # Test Percentage 
Takers 9th Grade Takers in9thGrade Takers in 9th Grade 
Spring 2011 313 63.6 - - - 7 
Gardens 
2012 379 72.0 401 55.4 389 40.6 
2013 432 72.7 539 44.3 348 26.4 
2014 424 77.4 445 38.0 509 22.0 
2015 452 77.9 401 32.4 480 21.3 
2016 287 85.4 449 18.9 439 17.1 
Total 2,287 74.8 2235 37.8 2165 24.9 
Hillview 2011 619 82.4 - - - - 
2012 679 83.8 815 44.2 664 40.2 
2013 556 89.7 664 44.3 290 24.8 
2014 538 95.4 683 48.6 638 16.9 
2015 612 87.7 641 50.4 635 19.5 
2016 529 80.0 714 41.9 568 19.2 
Total 3533 86.4 3517 45.7 2795 24.3 
Summers 2011 593 56.5 - - - = 
2012 497 66.2 280 70.7 526 21.5 
2013 499 68.9 551 36.5 329 11.6 
2014 624 69.6 573 24.6 628 10.8 
2015 574 71.4 603 21.7 496 21.8 
2016 377 89.9 579 33.3 685 17.1 
Total 3164 69.2 2586 33.4 2664 16.7 
Winter Ridge 2011 112 100.0 - - - - 
2012 120 96.7 333 99.4 228 54.8 
2013 87 98.9 274 99.3 206 44.2 
2014 105 75.2 338 95.6 244 55.7 
2015 117 79.5 327 97.2 229 65.9 
2016 61 95.1 295 97.3 233 66.5 
Total 602 90.4 1567 97:7 1140 57.7 
West 2011 403 87.3 - - - - 
2012 492 85.8 699 47.9 574 24.6 
2013 458 86.7 668 48.2 547 18.8 
2014 477 96.6 597 519 599 18.7 
2015 488 80.3 601 49.9 594 21.2 
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2016 455 96.9 706 53.7 623 30.0 
Total 2773 88.9 3271 50.3 2937 22.8 
Meadows 2011 660 82.6 - - 
2012 646 92.6 401 73.3 119 78.2 
2013 540 92.8 827 41.7 916 9.5 
2014 638 89.0 702 45.7 668 14.8 
2015 743 83.3 836 52.4 672 13.8 
2016 654 93.7 819 61.5 729 17.7 
Total 3881 88.7 3585 53.1 3104 16.1 
Coconut 2011 355 78.9 - - 
2012 304 67.4 277 23.8 284 7.0 
2013 276 76.1 318 26.1 186 11.8 
2014 295 54.6 363 17.6 333 10.2 
2015 441 67.8 602 59.1 282 17.7 
2016 106 61.3 88 46.6 348 9.2 
Total 1777 68.7 1648 37.0 1433 11.0 
White 2011 423 74.0 - - 
Mountain 
2012 504 80.0 536 39.2 448 15.8 
2013 482 83.4 544 38.6 465 17.0 
2014 254 91.3 484 40.5 511 25.0 
2015 438 47.9 516 32.9 458 31.0 
2016 427 47.5 547 23.2 422 27.5 
Total 2528 69.7 2627 34.8 2304 23.3 
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School Patterns in End-of-Course Test Taking Over Time 


Over the same period as the PASL rollout in Broward County, many schools changed the timing of 
end-of-course test taking. These changes were not uniform across the district or even among treated 
schools. For example, by the end of the 2016 school year 75% of Hillview High School students had 
taken the Biology End-of-Course test before high school, whereas less than 5% of students at Spring 
Gardens, Summers, and Meadows High Schools had done so. This variation in EOC timing across 
schools violates the parallel trends assumption of difference-in-differences models. 


According to the parallel trends assumption, treated and untreated groups have the same trends over 
time and differ only with respect to treatment status. The fact that some students in the district 
increasingly take the same exam at different ages from 2012 to 2016 means that we should expect 
differences in treated and untreated groups at the same time as treatment but for reasons unrelated 
to treatment. For this reason, the difference-in-differences estimates of the PASL intervention’s 
contribution to student achievement are biased in potentially arbitrary directions and magnitudes. 


The below tables illustrate the changes to the proportion of students taking an end-of-course exam 
before high school. Each percentage represents the entire high school’s proportion of students taking 
an exam in middle school, so a change from 0% to 5% in one year may represent a much larger change 
to the proportion of entering freshmen having taken an EOC in middle school. Any highly significant 
point estimates in the difference-in-difference analysis should be viewed in light of the changing 
proportions presented below to ensure that the estimate is not capturing selection bias from 
changing school policies and test-taking student populations. 
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Appendix Table A.13: Proportion of Pre-High School End-of-Course Test Takers 


School Year Algebra 1 Obs. Biology Obs. Geometry Obs. 
Spring Gardens 2011 0 1,934 0 0 0 0 
2012 8.58% 1,890 0 0 0 3,030 
2013 14.88% 1,945 2.08% 2,455 0 2,943 
2014 20.61% 2,005 2.51% 2,443 0.05% Ss 0n2) 
2015 25.22% 2,065 3.01% 2,533 0.11% 3,167 
2016 28.07% Zeal Sy7, 4.65% 2,488 0.48% Bh ley 
Hillview 2011 0 1,934 0 0 0 0 
2012 0 1,890 0 0 0 3,030 
2013 0 1,945 45.99% 1,152 2.65% 2,943 
2014 0.26% 2,005 55.32% dealgs 4.16% S002 
2015 2.93% 2,065 65.08% 1,192 6.01% 3,167 
2016 6.39% 2,157. 75.47% 1,202 9.49% Bh ile)7/ 
Summers 2011 0 1,934 0 0 0 0 
2012 0 1,890 0 1,250 2.71% AL Sil7/ 
2013 2.76% 1,945 0 1,152 5.11% 1,469 
2014 4.48% 2,005 0.35% 1,195 8.46% 1,338 
2015 6.54% 2,065 0.97% 1,192 12.85% 1,528 
2016 10.35% 2517, 4.94% 1,202 18.55% 1,449 
Winter Ridge 2011 0 3,274 0 0 0 0 
2012 5.80% 3,161 0 1,250 0 1,617 
2013 9.93% 2,871 14.19% 1,152 0 1,469 
2014 12.83% 2,882 19.70% 1,195 0 1,338 
2015 17.32% 3,000 24.32% 1,192 0.66% 1,528 
2016 20.58% 2,922 34.17% 1,202 3.23% 1,449 
Silver Oak 2011 0 3,274 0 0 0 0 
2012 0 3,161 12.64% 2,785 0 1,617 
2013 0.12% 2,871 18.45% 2,560 0.49% 1,469 
2014 0.27% 2,882 19.74% 2,700 0.53% 1,338 
2015 1.03% 3,000 25.79% 2,664 1.13% 1,528 
2016 1.99% 2,922 30.00% 2,739 2.76% 1,449 
Meadows 2011 0 3,274 0 0 0 0 
2012 0 3,161 0 72 TASS) 5.60% 2,174 
2013 2.31% 2,871 0 2,560 10.07% 2,118 
2014 2.80% 2,882 0 2,700 15.59% 2,207 
2015 4.16% 3,000 0.13% 2,664 23.17% 2,471 
2016 7.49% 2,922 0.38% 2739 29.80% 2,451 
Coconut 2011 0 2,692 0 0 0 0 
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2012 4.97% 2,564 0 2,785 0 2,174 
2013 7.91% 2,455 3.73% 2,560 0 2,118 
2014 10.69% 2,443 4.98% 2,700 0 2,207 
2015 15.58% 2,533 7.20% 2,664 0 2,471 
2016 20.89% 2,488 11.72% 2,739 0.59% 2,451 
White 2011 0 2,692 0 0 0 0 
Mountain 
2012 0 2,564 6.87% 3,030 0 Zralia 
2013 0 2,455 11.46% 2,943 0.90% 2,118 
2014 0.06% 2,443 14.54% 3,012 2.05% 2,207 
2015 1.39% 2,533 17.64% 3,167 3.94% 2,471 
2016 3.80% 2,488 20.07% 3,197 7.98% 2,451 
Year Algebra 1 Obs. Biology Obs. Geometry Obs. 
Never Treated 2011 0 49,922 0 0 0 0 
Schools 2012 8.29% 50,429 0 50,429 0 50,429 
2013 13.75% 46,186 0.09% 46,186 3.25% 46,141 
2014 17.68% 47,307 0.28% 47,307 4.79% 47,307 
2015 23.05% 49,652 1.41% 49,652 6.88% 49,652 
2016 27.83% 50,440 4.18% 50,440 11.27% 50,440 
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Appendix Table A.14: Proportion of Pre-High School Algebra | Test Takers by Grade 


2011 2012 2013 2014 2015 2016 
School Grade Pct. Obs. Pct. Obs. Pct. Obs. Pct. Obs. Pct. Obs. Pct. Obs. 
Pre- Pre-HS Pre-HS Pre-HS Pre-HS Pre-HS 
HS 
Spring Gardens High 9th 0.00% 515 16.18% 525 23.64% 598 28.72% 533 27.67% 587 33.60% 637 
School 
10th 0.00% 515 0.34% 447 16.19% 516 25.80% 563 27.54% 524 27.05% 561 
11th 0.00% 480 0.00% 437 0.35% 416 17.07% 492 25.86% 514 27.08% 466 
12th 0.00% 424 0.00% 481 0.00% 415 0.40% 417 17.89% 440 25.52% 492 
Hillview High School 9th 0.00% 804 11.71% 794 17.35% 636 18.84% 689 20.90% 783 25.92% 706 
10th 0.00% 879 0.00% 793 12.03% 756 17.35% 685 18.00% 742 20.14% 823 
1ith 0.00% 798 0.00% 815 0.00% 714 11.90% 769 17.40% 677 17.82% 724 
12th 0.00% 793 0.00% 759 0.00% 765 0.00% 739 12.36% 798 18.60% 669 
Summers High School 9th 0.00% 672 12.38% 609 12.88% 630 14.66% 613 24.69% 615 31.93% 585 
10th 0.00% 700 0.00% 680 12.36% 599 12.02% 692 13.52% 670 26.30% 636 
11th 0.00% 623 0.00% 683 0.00% 613 13.28% 563 11.13% 688 14.40% 600 
12th 0.00% 697 0.00% 592 0.00% 613 0.24% 575 13.16% 560 11.85% 667 
Winter Ridge High Schoo! | 9th 0.00% 350 43.01% 346 67.67% 273 72.88% 329 70.68% 334 82.37% 335 
10th 0.00% 322 0.00% 310 40.24% 321 67.08% 285 77.12% 294 72.10% 327 
11th 0.00% O 0.00% 282 0.00% 292 40.87% 294 66.81% 278 77.73% 269 
12th 0.00% O 0.00% 0 0.00% 0 0.00% 287 40.63% 286 67.83% 271 
Silver Oak High School 9th 0.00% 664 22.73% 738 28.42% 588 23.25% 654 31.38% 659 34.96% 746 
10th 0.00% 698 0.00% 702 22.39% 691 27.16% 667 22.94% 675 32.09% 688 
11th 0.00% 689 0.00% 673 0.00% 649 21.11% 719 26.42% 654 24.26% 664 
12th 0.00% O 0.00% 672 0.00% 632 0.00% 660 22.45% 676 27.60% 641 
Meadows High School 9th 0.00% 780 13.64% 835 19.58% 740 19.19% 764 16.32% 822 23.28% 863 
10th 0.00% 757 0.00% 759 14.16% 820 19.82% 796 19.12% 816 16.71% 813 
1ith 0.00% 747 0.00% 738 0.00% 720 14.22% 783 19.94% 770 18.55% 788 
12th 0.00% 702 0.00% 698 0.00% 663 0.00% 669 14.85% 759 21.63% 733 
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Coconut High School 9th 0.00% 513 6.11% 370 8.52% 423 12.76% 355 17.05% 489 42.98% 386 
10th 0.00% 555 0.00% 405 5.92% 349 11.28% 385 WAI  Sisy7/ 17.42% 429 
1ith 0.00% 496 0.00% 432 0.00% 324 5.75% 297 11.75% 391 14.04% 300 
12th 0.00% 511 0.00% 410 0.00% 373 1.02% 301 7.39%) 281 13.10% 334! 
White Mountain High 9th 0.00% 528 11.13% 610 18.74% 536 26.81% 593 33.04% 669 44.69% 598 
School 
10th 0.00% 569 0.00% 542 10.84% 570 19.25% 581 26.02% 635 31.94% 668 
1ith 0.00% 499 0.00% 537 0.00% 502 10.41% 573 19.28% 586 25.29% 627 
12th 0.00% 558 0.00% 485 0.00% 510 0.00% 460 12.39% 581 20.59% 558 
Never Treated Schools 9th 0.00% 12849 15.99% 13628 21.96% 11786 23.66% 12544 28.34% 13390 37.56% 13525 
10th 0.00% 13258 0.20% 12888 16.34% 12286 22.13% 12381 22.69% 13002 27.62% 13388 
11th 0.00% 11258 0.00% 12465 0.17% 10966 17.11% 11622 21.85% 11967 23.13% 12104 
12th 0.00% 9953 0.00% 10873 0.07% 11123 0.31% 10760 17.74% 11293 23.01% 11423 
Appendix Table A.15: Proportion of Pre-High School Biology Test Takers by Grade 
2012 2013 2014 2015 2016 
School Grade Pct. Obs. Pct. Pre-HS Obs. Pct. Obs. Pct. Obs. Pct. Pre- Obs. 
Pre-HS Pre-HS Pre-HS HS 
Spring Gardens High | 9th 0.00% 525 0.00% 598 0.90% 533 10.28% 587 37.04% 637 
Schoo! | 10th 0.00% 447 0.00% 516 0.00% 563 0.89% 524 8.42% 561 
11th 0.00% 437 0.00% 416 0.00% 492 0.00% 514 0.71% 466 
12th 0.00% 481 0.00% 415 0.00% 417 ~=0.00% 440 0.00% 492 
Hillview High School | 9th 0.00% 794 0.17% 636 0.65% 689 2.21% 783 5.68% 706 
10th 0.00% 793 0.14% 756 0.00% 685 1.39% 742 2.10% 823 
11th 0.00% 815 0.00% 714 0.30% 769 0.18% 677 1.51% 724 
12th 0.00% 759 0.00% 765 0.00% 739 0.15% 798 0.19% 669 
Summers High School | 9th 0.00% 609 0.00% 630 0.19% 613 5.43% 615 15.72% 585 
10th 0.00% 680 0.00% 599 0.00% 692 0.33% 670 5.88% 636 
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11th 0.00% 683 0.00% 613 0.00% 563 0.00% 688 0.39% 600 

12th 0.00% 592 0.00% 613. 0.00% 575 0.22% 560 0.00% 667 

Winter Ridge High School | 9th 0.00% 346 0.00% 273 0.92% 329 2.45% 334 =13.81% 0 
10th 0.00% 310 0.00% 321 0.00% 285 1.02% 294 2.80% 0 

11th 0.00% 282 0.00% 292 0.00% 294 0.00% 278 0.75% 0 

12th 0.00% 0 0.00% 0 0.00% 287 0.00% 286 0.00% 0 

Silver Oak High School | 9th 0.00% 738 0.00% 588 0.00% 654 0.34% 659 1.04% 746 
10th 0.00% 702 0.00% 691 0.00% 667 0.17% 675 0.48% 688 

11th 0.00% 673 0.00% 649 0.00% 719 0.00% 654 0.17% 664 

12th 0.00% 672 0.00% 632 0.00% 660 0.00% 676 0.00% 641 

Meadows High School | 9th 0.00% 835 0.00% 740 0.15% 764 0.28% 822 1.56% 863 
10th 0.00% 759 0.00% 820 0.00% 796 0.14% 816 0.42% 813 

11th 0.00% 738 0.00% 720 0.00% 783 0.00% 770 0.15% 788 

12th 0.00% 698 0.00% 663 0.00% 669 0.00% 759 0.00% 733 

Coconut High School | 9th 0.00% 370 0.00% 423 0.00% 355 1.79% 489 33.87% 386 
10th 0.00% 405 0.00% 349 0.00% 385 0.35% 367 2.28% 429 

11th 0.00% 432 0.00% 324 0.00% 297 0.00% 391 0.00% 300 

12th 0.00% 410 0.00% 373 0.00% 301 0.00% 281 0.00% 334 

White Mountain High | 9th 0.00% 610 0.00% 536 0.00% 593 0.00% 669 7.30% 598 
School | 10th 0.00% 542 0.00% 570 0.00% 581 0.00% 635 0.00% 668 

11th 0.00% 537 0.00% 502 0.00% 573 0.00% 586 0.00% 627 

12th 0.00% 485 0.00% 510 0.00% 460 0.00% 581 0.00% 558 

Never Treated Schools | 9th 0.00% 13604 0.06% 11786 0.69% 12544 5.02% 13390 19.69% 13525 
10th 0.00% 12888 0.19% 12269 0.11% 12381 0.73% 13002 4.67% 13388 

11th 0.00% 12465 0.00% 10965 0.17% 11622 0.08% 11967 0.68% 12104 

12th 0.00% 10295 0.00% 10595 0.00% 10760 0.17% 11293 0.09% 11423 
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Appendix Table A.16: Proportion of Pre-High School Geometry Test Takers by Grade 


2012 2013 2014 2015 2016 
School Grade Pct. Pre- Obs. Pct. Pre- Obs. Pct. Pre- Obs. Pct. Pre- Obs. Pct. Pre- Obs. 
HS HS HS HS HS 
Spring Gardens High School | 9th 0.00% 525 6.95% 598 8.61% 533 10.63% 587 40.94% 637 
10th 0.00% 447 0.00% 516 6.72% 563 7.57% 524 8.87% 561 
11th 0.00% 437 0.00% 416 0.00% 492 6.77% 514 6.71% 466 
12th 0.00% 481 0.00% 415 0.00% 417 0.00% 440 6.47% 492 
Hillview High School | 9th 0.00% 794 6.88% 636 4.55% 689 7.10% 783 28.76% 706 
10th 0.00% 793 0.00% 756 5.68% 685 4.61% 742 6.33% 823 
11th 0.00% 815 0.00% 714 0.00% 769 5.67% 677 4.01% 724 
12th 0.00% 759 0.00% 765 0.00% 739 0.00% 798 6.20% 669 
Summers High School | 9th 0.00% 609 6.26% 630 4.27% 613 3.66% 615 15.83% 585 
10th 0.00% 680 0.00% 599 4.68% 692 4.02% 670 3.38% 636 
11th 0.00% 683 0.00% 613 0.00% 563 3.94% 688 3.29% 600 
12th 0.00% 592 0.00% 613 0.00% 575 0.00% 560 4.17% 667 
Winter Ridge High School | 9th 0.00% 346 32.21% 273 29.68% 329 27.81% 334 44.84% 335 
10th 0.00% 310 0.00% 321 31.25% 285 31.71% 294 28.35% 327 
11th 0.00% 0 0.00% 292 0.45% 294 32.39% 278 30.68% 269 
12th 0.00% 0 0.00% 0 0.00% 287 0.00% 286 33.33% 271 
Silver Oak High School | 9th 0.00% 738 10.76% 588 7.90% 654 11.38% 659 26.38% 746 
10th 0.00% 702 0.00% 691 9.36% 667 7.67% 675 11.02% 688 
11th 0.00% 673 0.00% 649 0.00% 719 9.08% 654 7.90% 664 
12th 0.00% 672 0.00% 632 0.00% 660 0.00% 676 9.76% 641 
Meadows High School | 9th 0.00% 835 7.62% 740 7.74% 764 8.59% 822 34.18% 863 
10th 0.00% 759 0.00% 820 7.21% 796 7.60% 816 7.50% 813 
11th 0.00% 738 0.00% 720 0.00% 783 7.12% 770 6.17% 788 
12th 0.00% 698 0.00% 663 0.00% 669 0.00% 759 7.20% 733 
Coconut High School | 9th 0.00% 370 1.32% 423 0.90% 355 1.92% 489 23.81% 386 
10th 0.00% 405 0.00% 349 0.98% 385 1.21% 367 1.74% 429 
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11th 0.00% 432 0.00% 324 0.00% 297 1.02% 391 1.35% 300 

12th 0.00% 410 0.00% 373 0.00% 301 0.00% 281 1.64% 334 

White Mountain High School | 9th 0.00% 610 2.86% 536 4.94% 593 9.52% 669 29.27% 598 
10th 0.00% 542 0.00% 570 2.72% 581 4.69% 635 8.89% 668 

11th 0.00% 537 0.00% 502 0.00% 573 2.92% 586 4.55% 627 

12th 0.00% 485 0.00% 510 0.00% 460 0.00% 581 3.08% 558 

Never Treated Schools | 9th 0.00% 13604 9.10% 11768 8.77% 12544 11.37% 13390 8=©31.21% 13525 

10th 0.00% 12888 0.00% 12269 8.39% 12381 8.25% 13002 10.75% 13388 

11th 0.00% 12465 0.00% 10953 0.00% 11622 8.15% 11967 7.95% 12104 

12th 0.00% 11448 0.00% 11103 0.00% 10760 0.07% 11293 8.51% 11423 
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